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Abstract 

We propose a numerical method to approximate the value function for the optimal stop- 
ping problem of a piecewise deterministic Markov process (PDMP). Our approach is based 
on quantization of the post jump location - inter-arrival time Markov chain naturally embed- 
ded in the PDMP, and path-adapted time discretization grids. It allows us to derive bounds 
for the convergence rate of the algorithm and to provide a computable e-optimal stopping 



time. The paper is illustrated by a numerical example. 
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1 Introduction 



The aim of this paper is to propose a computational method for optimal stopping of a piece- 
wise deterministic Markov process {X(t)} by using a quantization technique for an underlying 
discrete-time Markov chain related to the continuous-time process {X(t)} and path-adapted 
time discretization grids. 

Piecewise-deterministic Markov processes (PDMP's) have been introduced in the literature 
by M.H.A. Davis [6] as a general class of stochastic models. PDMP's are a family of Markov 
processes involving deterministic motion punctuated by random jumps. The motion of the 
PDMP {X(t)} depends on three local characteristics, namely the flow <j), the jump rate A and 
the transition measure Q, which specifies the post-jump location. Starting from x the motion of 
the process follows the flow <f>(x, t) until the first jump time T\ which occurs either spontaneously 
in a Poisson-like fashion with rate \((f)(x,t)) or when the flow <p{x,t) hits the boundary of the 
state-space. In either case the location of the process at the jump time T\: X(T\) = Z\ is 
selected by the transition measure Q(<f)(x, T±), .). Starting from Z\, we now select the next 
interjump time T2 — T\ and postjump location Xi^T^) = Zi- This gives a piecewise deterministic 
trajectory for {X(t)} with jump times {T^} and post jump locations {Z^} which follows the 
flow <j> between two jumps. A suitable choice of the state space and the local characteristics <f>, 
A, and Q provides stochastic models covering a great number of problems of operations research 

Optimal stopping problems have been studied for PDMP's in El El 121 Ell E2|- In [H] the 
author defines an operator related to the first jump time of the process, and shows that the value 
function of the optimal stopping problem is a fixed point for this operator. The basic assumption 
in this case is that the final cost function is continuous along trajectories, and it is shown that the 
value function will also have this property. In [13] the authors adopt some stronger continuity 
assumptions and boundary conditions to show that the value function of the optimal stopping 
problem satisfies some variational inequalities, related to integro-differential equations. In [6], 
M.H.A. Davis assumes that the value function is bounded and locally Lipschitz along trajectories 
to show that the variational inequalities are necessary and sufficient to characterize the value 
function of the optimal stopping problem. In [5] , the authors weakened the continuity assuptions 
of [6j[9l[T3]. A paper related to our work is [3] by O.L.V. Costa and M.H.A. Davis. It is the only 
one presenting a computational technique for solving the optimal stopping problem for a PDMP 
based on a discretization of the state space similar to the one proposed by H. J. Kushner in [T2] . 
In particular, the authors in [3 derive a convergence result for the approximation scheme but 
no estimation of the rate of convergence is derived. 

Quantization methods have been developed recently in numerical probability, nonlinear fil- 
tering or optimal stochastic control with applications in finance [H [21 Q31 [T5l [TBI [T7] . More 
specifically, powerful and interesting methods have been developed in [TJ [21 [T7| for computing 
the Snell-envelope associated to discrete-time Markov chains and diffusion processes. Roughly 
speaking, the approach developed in [TJ [21 [TTj for studying the optimal stopping problem for a 
continuous-time diffusion process is based on a time-discretization scheme to obtain a 

discrete-time Markov chain {1^}. It is shown that the original continuous-time optimization 
problem can be converted to an auxiliary optimal stopping problem associated with the discrete- 
time Markov chain {1^}. Under some suitable assumptions, a rate of converge of the auxiliary 
value function to the original one can be derived. Then, in order to address the optimal stopping 
problem of the discrete-time Markov chain, a twofold computational method is proposed. The 
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first step consists in approximating the Markov chain by a quantized process. There exists an 
extensive literature on quantization methods for random variables and processes. We do not pre- 
tend to present here an exhaustive panorama of these methods. However, the interested reader 
may for instance, consult the following works |1 0^ IT4"1 [T7] and references therein. The second 
step is to approximate the conditional expectations which are used to compute the backward 
dynamic programming formula by the conditional expectation related to the quantized process. 
This procedure leads to a tractable formula called a quantization tree algorithm (see Proposition 
4 in [1] or section 4.1 in [IT]). Providing the cost function and the Markov kernel are Lipschitz, 
some bounds and rates of convergence are obtained (see for example section 2.2.2 in pQ). 

As regards PDMP's, it was shown in that the value function of the optimal stopping 
problem can be calculated by iterating a functional operator, labeled L (see equation ^ for 
its definition), which is related to a continuous-time maximization and a discrete-time dynamic 
programming formula. Thus, in order to approximate the value function of the optimal stopping 
problem of a PDMP {X(t)}, a natural approach would have been to follow the same lines as in 
[TJ |21 [T7| . However their method cannot be directly applied to our problem for two main reasons 
related to the specificities of PDMP's. 

First, PDMP's are in essence discontinuous at random times. Therefore, as pointed out in 
it will be problematic to convert the original optimization problem into an optimal stop- 
ping problem associated to a time discretization of {X(t)} with nice convergence properties. 
In particular, it appears ill-advised to propose as in [T] a fixed-step time-discretization scheme 
{X(kA)} of the original process {X(t)}. Besides, another important intricacy concerns the tran- 
sition semigroup {Pt}t^R + of {X(t)}. On the one hand, it cannot be explicitly calculated from 
the local characteristics (<fi,\,Q) of the PDMP (see [HE]). Consequently, it will be complicated 
to express the Markov kernel Pa associated to the Markov chain {X(kA)}. On the other hand, 
the markov chain {X(kA)} is in general not even a Feller chain (see [U pages 76-77]), therefore 
it will be hard to ensure it is iT-Lipschitz (see Definition 1 in [1]). 

Secondly, the other main difference stems from the fact the function appearing in the back- 
ward dynamic programming formula associated to L and the reward function g is not contin- 
uous even if some strong regularity assumptions are made on g. Consequently, the approach 
developed in [TJ [2j [T7] has to be refined since it can only handle conditional expectations of 
Lipschitz-continuous functions. 

However, by using the special structure of PDMP's, we are able to overcome both these 
obstacles. Indeed, associated to the PDMP {X(i)}, there exists a natural embedded discrete- 
time Markov chain {©&} with 0^ = (Zk,Sk) where S/. is given by the inter- arrival time 7\ — 
2fc_i. The main operator L can be expressed using the chain {©a,} and a continuous-time 
maximization. We first convert the continuous-time maximization of operator L into a discrete- 
time maximization by using a path-dependent time-discretization scheme. This enables us to 
approximate the value function by the solution of a backward dynamic programming equation 
in discrete-time involving conditional expectation of the Markov chain {©&}• Then, a natural 
approximation of this optimization problem is obtained by replacing {©&} by its quantized 
approximation. It must be pointed that this optimization problem is related to the calculation 
of conditional expectations of indicator functions of the Markov chain {©&}. As said above, it 
is not straightforward to obtain convergence results as in [U EJ [T7] . We deal successfully with 
indicator functions by showing that the event on which the discontinuity actually occurs is of 
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small enough probability. This enables us to provide rate of convergence for the approximation 
scheme. 



In addition and more importantly, this numerical approximation scheme enables us to pro- 
pose a computable stopping rule which also is an e-optimal stopping time of the original stopping 
problem. Indeed, for any e > one can construct a stopping time, labeled r, such that 

V(x)-e<E x [g(X(T))\ <V(x) 

where V{x) is the optimal value function associated to the original stopping problem. Our com- 
putational approach is attractive in the sense that it does not require any additional calculations. 
Moreover, we can characterize how far it is from optimal in terms of the value function. In [TJ 
section 2.2.3, Proposition 6], an other criteria for the approximation of the optimal stopping 
time has been proposed. In the context of PDMP's, it must be noticed that an optimal stopping 
time does not generally exists as shown in section 2]. 

An additional result extends Theorem 1 of U.S. Gugerli [11] by showing that the iteration of 
operator L provides a sequence of random variables which corresponds to a quasi-Snell envelope 
associated to the reward process {g(X(t))} t ^ where the horizon time is random and given by 

the jump time (T n ) n6 { ,...,iV} of the process {X(t)} t< _ R . 

The paper is organized as follows. In Section [2] we give a precise definition of PDMP's and 
state our notation and assumptions. In Section [3j we state the optimal stopping problem, recall 
and refine some results from [TT]. In Section |4j we build an approximation of the value function. 
In Section [5j we compute the error between the approximate value function and the real one. In 
Section [6] we propose a computable e-optimal stopping time and evaluate its sharpness. Finally 
in Section [7] we present a numerical example. Technical results are postponed to the Appendix. 



2 Definitions and assumptions 

We first give a precise definition of a piecewise deterministic Markov process. Some general 
assumptions are presented in the second part of this section. Let us introduce first some standard 
notation. Let M be a metric space. B(M) is the set of real- valued, bounded, measurable 
functions defined on M. The Borel a- field of M is denoted by B{M). Let Q be a Markov 

kernel on (M,B(M)) and w G B(M), Qw(x) = / w(y)Q(x,dy) for x G M. For (a, b) G M 2 , 

Jm 

a A b = min(a, b) and a V b = max(a, b). 



2.1 Definition of a PDMP 

Let E be an open subset of W 1 , dE its boundary and E its closure. A PDMP is determined by 
its local characteristics (<j), A, Q) where: 

• The flow (j) : M n x R — > M n is a one-parameter group of homeomorphisms: <p is continuous, 
0(., t) is an homeomorphism for each iGi satisfying </>(., t + s) = </>(</>(., s),t)). 
For all x in E, let us denote 

t*{x) = inf{t > : (j)(x,t) G dE}, 
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with the convention inf = oo. 

• The jump rate A : E — > R+ is assumed to be a measurable function satisfying: 



(Vx G -E 1 ), (3e > 0) such that f \(</>(x, s))ds < oo. 

Jo 



• Q is a Markov kernel on (E,B(E)) satisfying the following property: 

(Vx€S), Q(x,E- {x}) = 1. 

From these characteristics, it can be shown (6j p. 62-66] that there exists a filtered probability 
space (O, J 7 , {^i}, {P x }x£e) such that the motion of the process {X(t)} starting from a point 
x £ E may be constructed as follows. Take a random variable T\ such that 



e -A(a!,t) for t < 
for t > t*(x) 



P x (Ti > i) = 
where for x £ E and i 6 [0, t*(x)] 

A(x,t) = [ \(<j){x, s))ds. 
Jo 

If T\ generated according to the above probability is equal to infinity, then for t £ M + , = 
<f>(x,t). Otherwise select independently an .E- valued random variable (labelled Z\) having dis- 
tribution Q((f)(x,Ti), .). The trajectory of {X(t)} starting at x, for t < T\ , is given by 



X(t) = 



4>(x, t) for t < T±, 
Z\ for t = T\. 



Starting from X(T\) = Z\, we now select the next inter-jump time Ti — T\ and post-jump 
location X^T-i) = Z2 is a similar way. 

This gives a strong Markov process {X(t)} with jump times {7fc} fcgN (where To = 0). 
Associated to {X(t)}, there exists a discrete time process (©n) ngN defined by Q n = (Z n , S n ) 
with Z n = X(T n ) and S n = T n — T n _i for n > 1 and So = 0. Clearly, the process (0 n )neN is a 
Markov chain. 

We introduce a standard assumption, see for example equations (24.4) or (24.8) in [6]. 

Assumption 2.1 For all (x,t) £ E x R + , l^ Tk<t y < 00. 

fe 

In particular, it implies that T/% — > 00 as — > 00. 



For n E N, let A^ n be the family of all {J-i}-stopping times which are dominated by T n and 
for n < p, let A1 n) p be the family of all {^}-stopping times v satisfying T n < v < T p . Let B c 
denote the set of all real-valued, bounded, measurable functions, w defined on E and continuous 
along trajectories up to the jump time horizon: for any x € E, uu((f)(x, .)) is continuous on 
[0,£*(x)]. Let L c be the set of all real- valued, bounded, measurable functions, w defined on E 
and Lipschitz along trajectories: 
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1. there exists [w] 1 G R + such that for any (x, y) G E 2 , u G [0, t*(x) A t*(y)], one has 

\w(<t>(x,u)) - w((f>(y,u))\ < [w] x \x - y\, 

2. there exists [w] 2 G M.+ such that for any x e E, and (t, s) G [0, i*(x)] 2 , one has 

|u>(0(x,i)) - w((f>(x,s))\ < [w] 2 \t - s\, 

3. there exists [w] G M+ such that for any (x, y) G i? 2 , one has 

\w(4>(x,t*(x))) - w((p(y,t*(y)))\ < [w]Jx - y\. 

In the sequel, for any function / in B c , we denote by Cf its bound: 

C f = sup | f(x) |, 

a;S-B 

and for any Lipschitz-continuous function / in B(£') or B(£'), we denote by [/] its Lipschitz 
constant: 

[/] _ sup M, 

Remark 2.2 L c is a subset o/B c and any function in L c is Lipschitz on E with [w\ < [w\ 1 . 
Finally, as a convenient abbreviation, we set for any x G E, XQw(x) = X(x)Qw(x). 

2.2 Assumptions 

The following assumptions will be in force throughtout. 

Assumption 2.3 The jump rate A is bounded and there exists [A] 1 G M+ such for any (x, y) G 
E 2 , ue [0,t*(x) At*(y)[, 

|A(^(x,u))-A(^(y,u))| < [Ajjx-yl. 

Assumption 2.4 T/ie exit time t* is bounded and Lipschitz-continuous on E. 

Assumption 2.5 The Markov kernel Q is Lipschitz in the following sense : there exists [Q] G 
M+ such that for any function w G L c t/ie following two conditions are satisfied: 

1. for any (x,y) G -E 2 , u G [0,t*(x) At*(y)], one has 

\Qw((j)(x,u)) - Qw((/)(y,u))\ < [Q] [wj^x - y\, 

2. for any (x,y) G E 2 , one has 

\Qw{<t>{x,t\x)))-Qw{<l>{y,t*{y)))\ < [Q] [w]Jx-y\. 

The reward function g associated to the optimal stopping problem satisfies the following 
hypothesis. 

Assumption 2.6 g is in L c . 
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3 Optimal stopping problem 

From now on, assume that the distribution of X(0) is given by 5 XQ for a fixed state xq G E. Let 
us consider the following optimal stopping problem for a fixed integer N: 

sup E X0 [g(X(r))} (1) 
t&Mn 

This problem has been studied by U.S. Gugerli in 



Note that Assumption 2.3 yields A(x, t) < oo for all x, t. Hence, for all x in E, the jump time 
horizon s*(x) defined in [11] by t*(x) A inf{t > 0, e~ A ^ x '^ = 0} is equal to the exit time t*(x). 
Therefore, operators H : B(E) -> B(E x R + ), I : B(E) -> B(E x R + ), J : B(E) x B(E) -» 
B(E x R + ), K : B(E) -> B(E') and L : B(£) xB c ^B c introduced by Gugerli in [TTJ section 
2] reduce to 

If/fat) = f{^(x,tAf(x)))e- A ^ tM *^\ 

rtAt*(x) 

Iw(x,t) = / XQw((p(x,s))e- A{x ' s Us, 







J(wJ)(x,t) = Iw(x,t) + Hf(x,t), (2) 
ifw(x) = / \Qw{<j)(x,s))e- A{x > s) ds + Qw{<P{x,t* (x))) e - A(x '** (x)) , (3) 



J o 

L(w,h)(x) = sup J(w,h)(x,t) V Kw(x). 
t>o 

It is easy to derive a probabilistic interpretation of operators H, I, K and L in terms of the 
embedded Markov chain (Z n ,£ n ) N . 

Lemma 3.1 For all x G E, w G B(E), f G B(E), h G B c and i > 0, one /ias 

Hf(x,t) = f(<f>(x,tAt*(x)))P x (Si>tAt*(x)), 

Iw(x,t) = E x u;(Zi)l {Sl<tAt .( x )j 

Ku;(x) = E x [«;(Zi)], (4) 

L(w,h)(x) = sup {E a .[w(^i)l {Sl<1i} ]+/ l (0(x,u))P x (5i>«)jvE !B [«;(Zi)]. (5) 

u<t*(x) 1 J 



For a reward function g G B c , it has been shown in [TT] that the value function can be 
recursively constructed by the following procedure: 

sup E Xo [g(X(r))} = v (xo) 

t<eMn 

with 

\VN = 9, 

\v k = L(v k+1 ,g) for fc < AT - 1. 
Definition 3.2 Introduce the random variables (V n ) n &{o,...,N} 

V n = v n (Z n ), 
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or equivalently 



V n = sup {e v n+1 (Z n+ i)l {Sn+1<u} + g(<p(Z n ,u))l {Sn+1 > u} \Z n } 

u<t*(Z n ) 11 J > 



V E 



v n +i(Z n +i)\Z n 



(6) 



The following result shows that the sequence (Ki)ne{o,...,7V} corresponds to a quasi-Snell 
envelope associated to the reward process {5 , (^(^))} fg K + where the horizon time is random and 
given by the jump times (2 n ra) n e{o....,7V} °f the process {A(i)} tgK+ : 
Theorem 3.3 Consider an integer n < N. Then 



Vn 



sup E Xo [g{X{v))\F Tn }. 



Proof: Let v € M n ,N- According to Proposition B.4 and Corollary B.6 in Appendix |Bj 
there exists v: E x (R + x E) n x O R + such that for all (20,7) £ E x (R + x E) n the 
mapping P( 210,7): fi — > R+ is an {j^} tgR+ -stopping time satisfying 9(zq,j) < Tjv_ n , and ^ = 

?n + ^(^0; T n , #T n )> where T n = (Si, Z±, . . . , 5 n , Z n ) and 9 is the shift operator. For (zo, 7) S 
E x (M + x £) n define W:£x (R+ x £) n -» M by 

W(z ,7) = E 2n [<7(X(P(z ,7)))] < sup 

where 7 = (si, 21, . . . , s n , z n ). Hence, the strong Markov property of the process {X(t)} yields 

E X0 [g(X(v))\fT n ] =E X0 [g(X(Tn + HZ ,T n ,e Tn )))\F Tn } =W(Z ,T n ). 
Consequently, one has 

E X0 [g(X(v))\FT n ] < sup E z MX(r))], 

T&M N - n 



and therefore, one has 



sup E X0 [g{X(y))\F Tn ] < sup E Zn [g(X(r))], 

V&M n ,N T&Mn-ti 



(7) 



Conversely, consider r £ Ai^-n- It is easy to show that T n + r o Tn £ M n ,N- The strong 
Markov property of the process {X(t)} again yields 

E Zn [g(X(T))}=-E X0 [g(X(Tn + Toe Tn ))\F Tn }< sup E Xo [g(X^))\^T n ], 

v&M n , N 



hence we obtain 



sup E Zn [g(X(T))} < sup E X0 [g(X(v))\F T J. 



(8) 



Conbining equations ((?]) and pj), one has 



sup E Zn [g(X(r))} 

TeM N -n 



sup E X0 [g{X( V ))\F Tn ] 



Finally, it is proved in [TT| Theorem 1] that v n {x) = sup r6 ^ E x [g(X(r))], whence 

V n = sup E Zn [g(X(r))], 

reMff-n 



showing the result. 



□ 
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4 Approximation of the value function 



To approximate the sequence of value functions (V n ), we proceed in two steps. First, the 
continuous-time maximization of operator L is converted into a discrete-time maximization by 
using a path-dependent time-discretization scheme to give a new operator L d . In particular, it is 
important to remark that these time-discretization grids depend on the the post-jump locations 



{Z k } of the PDMP (see Definition 4.1 and Remark 4.2). Secondly, the conditional expectations 
of the Markov chain (@ k ) in the definition of L d are replaced by the conditional expectations of 
its quantized approximation (Ofc) to define an operator L d . 



First, we define the path-adapted discretization grids as follows. 



Definition 4.1 For z G E, set A(z) &]0,t*(z)[. Define n{z) = ^(^jy) ~~ 1> where int(x) 
denotes the greatest integer smaller than or equal to x. The set of points (£i)ig{o,...,n(z)} with 
ti = iA{z) is denoted by G{z). This is the grid associated to the time interval [0,t*(z)]. 

Remark 4.2 It is important to note that, for all z G E, not only one has t*(z) ^ G{z), but also 
maxG(z) = t n r z \ < t*(z) — A(z). This property is crucial for the sequel. 

Definition 4.3 Consider for w G B(i?) and z G E, 



L d (w,g)(z) 



max 

seG(z) 



|E w(Zi)l {Sl<s} + g((p(y,u))l {Sl > u} \Zo = z } 



V E 



w(Zi)\Z 



Now let us turn to the quantization of (0„). The quantization algorithm will provide us 
with a finite grid r® C E x M + at each time < n < N as well as weights for each point of the 
grid, see e.g. [U Q31 [T7] . Set p > 1 such that O n has finite moments at least up to the order p 
and let p n be the closest-neighbour projection from E x IR + onto T® (for the distance of norm 
p; if there are several equally close neighbours, pick the one with the smallest index). Then the 
quantization of Q n is defined by 

@n = \Z n , Snj =p n (Z Tll S n ). 

We will also denote T% the projection of T® on E and the projection of T® on M + . 

In practice, one will first compute the quantization grids and weights, and then compute a 
path-adapted time-grid for each z G T^, for all < n < N — 1. Hence, there is only a finite 
number time grids to compute, and like the quantization grids, they can be computed and stored 
off-line. 

The definition of the discretized operators now naturally follows the characterization given 
in Lemma I3JJ 



Definition 4.4 For k G {1, . . . , N}, w G B(rf), z G Tf_ 1; and s G M+ 

J k {w,g){z,s) = E w(Z k )l0 k<s} +g[(j){z, s))l { £ fc > s} |Z fe _i = z 
K k (w)(z) = E w(Z k )\Z k _i = z 



L d k {w,g){z) 



max 



{Jk(w,g)(z,s)} V K k (w)(z). 
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Note that O n is a random variable taking finitely many values, hence the expectations 
above actually are finite sums, the probability of each atom being given by its weight on the 
quantization grid. We can now give the complete contruction of the sequence approximating 
(K). 

Definition 4.5 Consider vn(z) = g(z) where z G Y N and for k G {1, . . . , N} 

v k -!(z) = L d k (y k ,g)(z), (9) 

where z G r?_ 1 

Definition 4.6 The approximation of V k is denoted by 

% = v k (Z k ), (10) 

forke {0,...,N}. 

5 Error estimation for the value function 

We are now able to state our main result, namely the convergence of our approximation scheme 
with an upper bound for the rate of convergence. 

Theorem 5.1 Set n G {0, . . . , N — 1}, and suppose that A(z), for z G T^, are chosen such that 
i{A(z)} > (2C A )- 1 / 2 ([t*] \\Z n - Z n \\ p + \\S n+1 - 5 n+ i|| p ) 1/2 . 

Then the discretization error for V n is no greater than the following: 

\\V n — V n \\ p < \\V n+ i — V n+ i\\ + a||A(Z n )|| p + n \\Z n — Z n \\ p + 2[u n +i] \\Z n+ \ — Z n+ i\\ p 

+t( [**] \\Zn — Z n \\ p + \\S n +i — Sn+iWp) 1 ^ 2 , 

where a= [g] 2 + 2C g C x , (3 n = [v n ] + K+i]^ + C g E 4 +{[g] 1 + [g] 2 [f]) V ( [v n+1 ] # [Q] ) , 
7 = AC g {2C\) 1 / 2 , and E2 and E± are defined in Appendix \a\ 

Recall that Vn = g{Zj<i) and Vn = 9(Zn), hence ||Vjv — Vw\\p < [g] \\Zn — Zn \\ • In addition, 
the quantization error ||G n — n || p goes to zero as the number of points in the grids goes to 
infinity, see e.g. [T3]. Hence | Vo — Vo | can be made arbitrarily small by an adequate choice of 
the discretizations parameters. 

Remark that the squere root in the last error term is the price to pay for integrating non- 
continuous functions, see the definition of operator J with the indicator functions, and the 
introduction of section 15.21 



mm- 



To prove Theorem 5.1, we split the left-hand-side difference into four terms: 

4 



IK -Klip < Y, E 



It 

1=1 
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where 

Hi = \\v n (Z n ) - v n (Z n )\\ p 

S 2 = \\L(v n+1 ,g)(Z n ) - L d (v n+ i,g)(Z n )\\ p 

Z 3 = \\L d (v n+1 ,g)(Z n )-Li +1 (v n+1 ,g)(Z n )\\ p 

S 4 = H^+^n+l^X^n) -L d +1 {v n+1 ,g)(Z n )\\ p . 
The first term is easy enough to handle thanks to Proposition |A.7| in Appendix A. 2 
Lemma 5.2 A upper bound for Si is 

\\v n {Zn) ~ v n (Z n ) ||p < [v n ] \\Z n — Z n \\ p . 
We are going to study the other terms one by one in the following sections. 

5.1 Second term 

In this part we study the error induced by the replacement of the supremum over all non-negative 
t smaller than or equal to t*(z) by the maximum over the finite grid G{z) in the definition of 
operator L. 

Lemma 5.3 Let w E L c . Then for all z 6 E, 

j sup J(w,g)(z,t) - max J(w,g)(z,s)\ < (C W C\+ [g] + C g C\) A(z). 

t<t*(z) seG(z) 

Proof: Clearly, there exists t G [0,i*(z)] such that sup t<t ,^ J (w, g)(z,t) = J(w, g)(z,t), and 
there exists < i < n(z) such that t £ [ti, ti+i] (with i„( z )+i = t*(z)). Consequently, Lemma 
yields 

< sup J(w,g)(z,t) - max J(w,g)(z,s) < J(w, g)(z,t) - J(w, g)(z,ti) 

t<t*(z) seG(z) 

< {C W C X + [g] 2 + C g C x )\i-ti\ 

< (C W C X + [g] 2 + C g C x ) ~ U\, 
implying the result. □ 

Turning back to the second error term, one gets the following bound. 

Lemma 5.4 A upper bound for H2 is 

\\L(v n+1 ,g)(Z n ) - L d {v n+1 ,g)(Z n )\\ P < ([<?] 2 + 2C g C x ) || A(Z n )||„. 

Proof: From the definition of L and L d we readily obtain 

\\L(v n+ i, g)(Z n ) - L d {v n+1 ,g)(Z n )\\ p 

< || sup J(v n+1 ,g)(Z n ,t) - max J(v n+1 , g)(Z n , s)\\ . 
t<t*(z n ) seG(z n ) 



A.2 



Now in view of the previous lemma, one has 
\\L(v n+1 ,g)(Z n ) - L d (v n+ iig){Z, 



n) ||p 

< (C Vn+1 Cx+ [g} 2 + C g Cx)\\A(Z n )\\p. 



Finaly, note that C Vn+1 = C g (see Appendix |A.2), completing the proof. □ 
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5.2 Third term 

This is the crucial part of our derivation, where we need to compare conditional expectations 
relative to the real Markov chain (Z n , S n ) and its quantized approximation (Z n , S n ). The main 
difficulty stems from the fact that some functions inside the expectations are indicator functions 
and in particular they are not Lipschitz-continuous. We manage to overcome this difficulty by 
proving that the event on which the discontinuity actually occurs is of small enough probability, 
this is the aim of the following two lemmas. 

Lemma 5.5 For all n € {0, . . . , N — 1} and < t] < min{A(2;)}, 



max E[|l 

s£G(Z n ) 



{S n+1 <s} 1 {S n+1 <s}\\ Zn 



< -\\S n+ i — S n+ i\\ p + 2C\rj. 
v V 



Proof: Set < r\ < min{A(;z)}. Remark that the difference of indicator functions is non-zero 

2 erf 

if and only if 5 n +i and S n +i are on either side of s. Hence, one has 



{S n+1 <s} - L {5„ +1<S }I ^ 1 {|S„ + i-S n +ll>'7} i ~ MlSn+i-sl^}- 



This yields 



max E[\l {Sn+1<s} -l {dn+i<s} \\Z n ] 



< 



"{|5 n +i-S'„+i|>2r;} Hp 



+ || max E[l 



s£G(Z n ) 

On the one hand, Chebychev's inequality yields 



{s-r)<S n+1 <s+r)} \Zn 



(11) 



"{|5 n+ i-S n+ i|>r;} Hp 



P(|5 n+ i - 5n+i| > v) < 



I Q Q I IP 

\On+l — dn+l\\ p 
rf ' 



(12) 



On the other hand, as s G G{Z n ) and by definition of rj, one has s + r] < t*(Z n ), see Remark 
Thus, one has 



4.2 



E[l 



{s-r)<S n+1 <s+r)}\ Z n 



= E 

= E 

< 2 V C X . 



E [' 1 -{s-Ti<S n+ i<s+r]}\ Z n — Z n] 



s+r) 



\(4>(Z n ,u))du 



s—rj 



Z n 



Combining equations ( 11 )-{ 13 ) , the result follows. 



(13) 

□ 



Lemma 5.6 For all n G {0, . . . , N} and < r/ < min {A(z)} ; 



l 1 t*(Z n )<t*(Z„)-^llp - 



[t*] \\Z n — Z n \ 
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Proof: We use Chebychev's inequality again. One clearly has 



E 



1 \ p 

|-V(£„)<i*(Z„)-?7l 



P[t*(Z n )<t*(Z n )-ri 



< p( t*(z k )-t*(z k )\ > v < 



[t*Y\\z k -z k f p 



showing the result. 



□ 



Now we turn to the consequences of replacing the Markov chain (Z n ,S n ) by its quantized 
approximation (Z n ,S n ) in the conditional expectations. 

Lemma 5.7 Let w G L c , then one has 
E[w{Z n+l )\Z n = Z n ] -E[w(Z n+1 )\Z n 

Z n +\ — Z n+ \ I Z n I . 

Proof: First, note that 

E[w(Z n+1 )\Z n = Z n ] -E[w{Z n+1 )\Z n ] = E[w(Z n+ i)\Z n = Z n ) - E[w(Z n+1 )\Z n ] 

+ E[w{Z n+l )\ Z n ] - E [w(Z n+1 ) | Z n ] . 

On the one hand, Remark |2.2| yields 

E[w(Z n+1 )\Z n ] -E[w(Z n+1 )\Z n ] \ < [w]E[\Z n+1 -Z n+1 \\Z n ]. 

On the other hand, recall that by construction of the quantized process, one has (Z n , S n ) = 
p n (Z n , S n ) . Hence we have the following property: a{Z n } C a{Z n ,S n }. By using the special 
structure of the PDMP {X(t)}, we have cr{Z n , S n } C Tx n . Now, by using the Markov property 
of the process {X(i)}, it follows 



Z r , 



E 



E[w{Z n+1 )\Z n ] =E\E[w(Z n+l )\F Tn 

Equation Q thus yields 

E[w(Z n+ i)\Z n = Z n ] - E[w(Z n+1 )\Z n 



E[w{Z n+x )\Z n ] 



E 
E 



E[w(Z n+1 )\Z n = Z n ] - E[w(Z n+1 )\Z n 



Zn 



Kw{Z n ) - Kw{Z n 

Now we use Lemma lA.4l to conclude. 

Now we combine the preceding lemmas to derive the third error term. 
Lemma 5.8 For all < r/ < min{A(z)} ; an upper bound for H3 is 

\\L d (v n+1 ,g)(Z n ) - Li +1 {v n+l ,g){Z n )\\ p 

[t* 



□ 



< 



{ [v n+1 ] X E 2 + C g E 4 + 2C g LA + ( [ g ] x + [ g ] 2 [ t *] ) v ( [v n+1 ] JQ] ) } \\Z n - Z n I 



+ [v n+ i] \\Z n+ i - Z n+ i\\ p + 2C g [2C\r] + 



V 
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Proof: To simplify notation, set ty(x,y,i) = v n+ i(y)l^ t<s y + g((j){x, t))lr t > s \. From the defini- 
tion of L d and L d +1 , one readily obtains 

| L d (v n+1 , g) (Z n ) -L d n+1 (v n+ i , g) (Z n ) I 



< max 

seG(z n ) 



V 



E[y(Z n ,Z n+1 ,S n+l )\Z n = Z n ] -E[*(Z n > Z n+ i, S n +\)\Z n 

E[vn+l(Zn+l)\Z n = Z n ] — E[f rt+ i(Z n+ i)|Z n ] 



(14) 



On the one hand, combining Lemma 5.7 and the fact that u n +i is in L c (see Proposition A.7), 
we obtain 

E[vn+i(Z n +i)\Z n = Z n ] — F,[v n+ i(Z n+ i)\Z n ] 

< [fn+l]E[|Z n+ i — Z n+ \ I Z n ] 

+ {C g E± + [vn+x]^ + [v n+1 ] JQ])E[|Z„ - Z n \ | Z n ]. (15) 
On the other hand, similar arguments as in the proof of Lemma |5.7| yield 
E[^(Z n , Z n+ i, S n+ \)\Z n = Z„] — E[^(Z n , Z n+ i, S n+ \)\Z n \ 

= E E[^(Z n , Z n+ i, S'„ + i)|Z n = Z n ] — E[^(Z„, Z n+ i, 5 n+ i)|Z n = Z n ] Z n 

+E[tf(Z n ,Z n+ i,S„ + i)|Z n ] -E[^(Z n ,Z n+ i,5 n+ i)|Z„]. (16) 
= Ti + T 2 



The second difference of the right hand side of equation (16), labeled T2, clearly satisfies 
|T 2 | < [v n+1 ]B[\Z n+1 -Z n+1 \\Z n ] + [gj^Zn-ZnllZn] 



+2C E II 



{S n+ i<s} i {S„+i<s}l 



(17) 



Let us turn now to the first difference of the right hand side of equation (16), labeled Ti. We 
meet another difficulty here. Indeed, we know by construction that s < t*(Z n ), but we know 
nothing regarding the relative positions of s and t*{Z n ). On the event where s < t*{Z n ) as well, 
we recognize operator J inside the expectations. On the opposite event s > t*(Z n ), we crudely 
bound ^ by C Vn+1 + C g = 2C 9 . Hence, one obtains 



|Ti 



< E 



I J(v n+ x,g)(Z n ,s) - J(^ + i,#)(Z n ,s)|l {s < t *(z n )} Z n +2C fl E[l{ t *(2 n ) <s }|Z n ]. 



Now Lemma |A.3| gives an upper bound for the first term. As for the indicatror function, by 
definition of G{Z n ) and our choice of r/, we have s < t*{Z n ) — 77. Thus, one has 



|Ti| < (C g E l +[v n+1 ] 1 E 2 + E 3 )E[\Z n -Z n \\Z n ]+2C g E[l 



{t*(Z n )<t*{Z n )-v}\ Zn 



(18) 



Now, combining (14), (15), (17) and (18), and the fact that C g Ex + E 3 = C g E 4 + [g] 1 + [g] 2 [t*] 
one gets 

\L d {v n+l ,g){Z n ) - L d n+l (v n+1 ,g)(Z n )\ 

< [{v n+l ] x E 2 + C g E i +{[g] l + [g) 2 [t*]) V ([ Un+1 ]jQ])}E[|Z n -Z n ||Z n ] 
1 — Z n+ \\ |z n ] 



+ 2C 9 E[1 



Z n ]+2C g max E[|l {Sn+1<s} 

S&G{Z n ) 



L {S n+1 <s}\ 



Finally, we conclude by taking the LP norm on both sides and using Lemmas 5.5 and 5.6 



□ 
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5.3 Fourth term 

The last error term is a mere comparison of two finite sums. 
Lemma 5.9 An upper bound for H4 is 

\\L^ +1 (v n+1 ,g)(Z n ) -L^ +1 (v n+ i,g)(Z n )\\ p < [v n+1 ]\\Z n+1 - Z n+1 \\ + \\V n+1 -V n+1 \\ 



Proof: By definition of operator L^, one has 



\\ L n+l( v n+l,g){Z n ) ~ L n+l (v n+1 , g) (Z, 

max < E 

seG(z n ) l 



nj\\p 



d ) '„, „\ { ry \ j d 

v n+1 (Z n+1 )l {§n+i<s} + g(^(Z n ,s))l {§n+i ^ s} \Z n 
v n+1 (Z n+1 )l { 3 n+i<s} + g(<l>(Z n ,s))l {§n+i > s} \Z n 

< \\E[v n+ i(Z n+ i) - v n+ i(Z n+ i)\Z n ]\\ 

< |pn+l(-^n+l) — v n +i(Z n +i)\\ p + \\v n +i(Z n +i) — v n+ i(Z n+ i)\\ p . 



max i E 

seG(z n ) 



V E 



Vn+l{Z n +l)\Z; 
V n +l(Zn+l)\Z n 



We conclude using the fact that i> n +i G L c (see Proposition A. 7) and the definitions of V n and 
V n . □ 

5.4 Proof of Theorem 15.11 

We can finally turn to the proof of Theorem |5.1[ Lemmas 5.2 5.4 5.8 and 5.9 from the preceding 
sections directly yield, for all < ?? < min{A(z)}, 

\\V n - V n \\ p < [v n ] \\Z n - Z n \\ p + ([g] 2 + 2C g C x )\\A(Z n )\\ p 

\t*] 

+ { [v n+1 ] X E 2 + C g E 4 + 2C g ^ + ( [g] l + [g] 2 [t*] ) V ( [v n+1 ] JQ] ) } \\Z n - Z n \\ p 



+ [v n+1 ] \\Z n+1 - Z n+1 \\ p + 2C g [2CxV + 
+ [ u n+l] \ \Z n +l — Z n+ \\\ p + ||V^+i — V n +i \\ p . 
The optimal choice for r\ clearly satisfies 

1 ~ ~ 

2Ca?? = -([**] ||-^n — ^n[|p + ll^n+l — Sn+l||p), 

providing it also satisfies the condition < rj < min{A(z)}. Hence, rearranging the terms 
above, one gets the expected result: 

\\V n -V n \\ p <\\V n+l -V n+1 \\ p + ([g] 2 + 2C g C x )\\A(Z n )\\ p 

+ { [v n ] + [v n+1 ] X E 2 + C g E A +([g} 1 + [g] 2 [f] ) V ( [v n+1 ] JQ] ) } || Z n - Z 



Tl IIP 



+ 2[v n+1 ] \\Z n+1 - Z n+1 \\ p + 4C g (2C x ) 1 / 2 {[t*] \\Z n - Z n \\ p + 



ra+l 



'S'n+l ||p 



»V2 
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6 Numerical construction of an e-optimal stopping time 



In |1 1 1 Theorem 1], U.S. Gugerli defined an e-optimal stopping time for the original problem. 
Roughly speaking, this stopping time depends on the embedded Markov chain (O n ), and on the 
optimal value function. Therefore, a natural candidate for an e-optimal stopping time should 
be obtained by replacing the Markov chain (0 n ) and the optimal value function by their quan- 
tized approximations. However, this leads to un-tractable comparisons between some quantities 
involving (0 n ) and its quantized approximation. It is then far from obvious to show that this 
method would provide a computable e-optimal stopping rule. Nonetheless, by modifying the 
approach of U.S. Gugerli [IT], we are able to propose a numerical construction of an e-optimal 
stopping time of the original stopping problem. 

Here is how we proceed. First, recall that p n be the closest-neighbour projection from E x 
onto r®, and for all (z, s) G E x M + define (z n , s n ) = p n (z, s). Note that z n and s n depend on 
both z and s. Now, for n G {1, . . . , N}, define 

s* n {z,s) = mmlt G G(z n -\) J n (v n ,g)(z n -i,t) = max J n {v n , g)(z n -\, u) \ 

«6G(2„_l) > 



and 



{t*(z) if K n v n (z n -i) > max J n (v n ,g)(zn-i,u), 
<0, s)l {s *( Z)S ) <r ( z ) } + (i*0) - /3)l{ s *( Z)S )> t *( z )} otherwise. 

Note the use of both the real jump time horizon t*{z) and the quantized approximations of K, 
J and (z, s). Set 

n = r N) p(Z ,S ) AT 1} 

and for n G {1, . . . , N — 1}, set 



Tn+l 



rN-n,/3(Z , S ) if Tl > r N _ nt /3(Z , S ) 

T\ + T n o Tl otherwise. 



Our stopping rule is then defined by t^. 

Remark 6.1 This procedure is especially appealing because it requires no more calculation: we 
have already computed the values of K n and J n on the grids. One just has to store the point 
where the maximum of J n is reached. 

Lemma 6.2 rjy is an {J-t} -stopping time. 

Proof: Set U% = r lt/3 (Z ,S ) and for 2 < k < N U k = r ki p{Z k _ u <S , fc-i)l{r fc _ lii8 (g fc _ a ,g fc _ 2 j>gfc_ 1 } 

N 

One then clearly has ttv = U k A S k which is an {jT T }-stopping time by Proposition 



fc=i 



B.5 



□ 



Now let us show that this stopping time provides a good approximation of the value function 
Vq- Namely, for all z G E set 

v n (z) = E[g(X TN _ n )\Z n = z], 
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and in accordance to our previous notation introduce, for n G {1, . . . , N — 1} 

V n = v n {Z n ). 

The comparison between Vq and Vq is provided by the next two Theorems. 



Theorem 6.3 Set n G {0, . . . , N — 2} and suppose the discretization parameters are chosen 
such that there exists < a < 1 satisfying 



'- = (2C x r 1/2 (^\\Z n -Z n \\ p + \\S n+1 -S n+1 \\ p ) ' <min{A(z)}. 
Then one has 

\\Vn — Vn\\ p < ||^n+l — ^n+i|| + ||^4+1 — ^ra+l|| p + \\Vn ~ Vn\\ p 
v n+l] \\Z n +l — Zn+l\\ p + a n\\Z n — Z n \\ p 

+AC g (2C\) 1 l 2 ^ ll-^Tt — ^ra||p + ||Sn+l — Sn+lHp) > 

with a n = (2 [ Un+1 ] ± E 2 + 2C 9 C t . [A] 1 (2 + C t *C x ) + (4C S C A [f] + 2 [« n+1 ] JQ] ) V (3 [g] x )) . 
Proof: The definition of r n and the strong Markov property of the process {X(t)} yield 

V n {Z n ) 

= E [5 , (^r n+li(3 (Z n ,5 Il )) 1 {5 n+1 >r n+li(3 (Z„,5 n )}|^n] + ^\v n+ i{Z n+1 )l {Sn+1 < rn+1 ^ ZnjSn) ^\Z n ] 
= 1 {r n+hl} (Z n ,S n )>t*(Z n )}KVn+l(Zn) + 1 {r n+li0 {Z n ,S n )<t*(Z n )}J(v n +l, 9){Z n , r n +l,p{Z n , S n )). 

However, our definition of r„ )( g with the special use of parameter (3 implies 

{r n+ i t p(Z n ,S n ) > t*{Z n )\ = \ k n+1 v n+ i(Z n ) > max J n+1 (v n+1 , g)(Z n , s)\ . 

1 seG(z n ) > 

Consequently, one obtains 



N 1/2 



v n (Z n ) = K n+1 v n+1 (Z n ) V max J n+ i(£ n+ i, c/)(Z n , s) 

seG(z„) 



+ 1 {r-n+l,/3(2„,5 n )>t*(Z n )} 

+ 1 {r Il+ i,^(Z n ,5 n )<i*(Z„)} 



Kv n+ i(Z n ) - K n+1 V n+ i(Z n ) 



J(v n +i,g)(Z n ,r n+ i ! i 3 (Z n , S n )) - max J n+ i(t; n+ i, s) 

seG(z n ) 



(19) 



Let us study the term with operator K. First, we insert V n to be able to use our work of the 
previous section (we cannot directy apply it to v n because it may not be Lipschitz-continuous). 
Clearly, one has 



Kv n+ i(Z n ) - K n+1 v n+1 (Z n ) 



< E||V n +i — V n+ i\\Z n 



+ 



Kv n+ i{Z n ) - K n+ iv n+ i{Z n 



(20) 
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Similar calculations to those of Lemmas A.4 5.7 and 5.9 and Equation (15) yield 
Kv n+ i(Z n ) - K n+ iv n+ i(Z n ) 



< {C g E 4 + [v n+1 ] 1 E 2 + [vn+i] m [Q])[\Z n -Z n \ +B[\Z n - Z n \\Z r 
+2[v n+ i]E[|Z n+ i — Z n +i\\Z n ] + E [| — V n +i\ \Z n ] . 

Now we turn to operator J. Set R n = r n+ i^(Z n , S n ). We first study the case when R r , 
s n+1 (Z n , S n ) < t*(Z n ). By definition, one has 

Jn+i{v n +i,g){Z n ,Rn) = max J n+1 (v n+ i,g)(Z n ,s). 

s&G(Z n ) 

As above, we insert V n and obtain 



(21) 



J(v n+ x,g)(Z n ,R n ) - max J n+1 (v n +i, g)(Z n , s) 

s£G(Z n ) 

< E[\V n +l ~ V n +l\\Z n ]l {Rn=s * n+i ( ZntSn) y 



: -{R n =s* n+1 (Z n ,S n )} 



+ 



J(v n+1 ,g)(Z n ,R n ) - J n+1 (v n+1 ,g)(Z n ,R n )) l{R n = s * n+1 (z n ,s n )}- (22) 



Again, similar arguments as those used for Lemmas A. 3 5.6 and 5.9 and Equations (16), (17) 
and ([l8| yield, on {Rn = s* n+1 (Z n , S n )} 

J{v n +l,g){Z n ,Rn) ~ Jn+l{Vn+l, g){Z n , Rn)) 

< ( [v n+1 ] x E2+[g] x + C g C t > [X] 1 (2 + Ct*Cx))(\Z n -Z n \+B[\Z n -Z n \\ Z r 

+2[u n +i]E[|z ri ,-i_i — z n+ i\\z n ~\ + E[|y ra+ i — y n+ i||z n ] 

+ [g]^[\Z n - Z n \\Z n }+2C g E[\l {Sn+1<Rn} - l 0n+i<Rn} \\Z n ]. (23) 

Note that all the constants with a factor [t*] have vanished, because we know here that both 
R n < t*(Z n ) and R n < t*(Z n ) hold on {R n = s n+l (Z n , S n )}. 



Finally, on {s*(Z n ) > t*(Z n ) = R n + (3}, by construction of the grid G(Z n ) (see Remark 4.2), 
one has for all < r/ < min{A(2;)}, 

R n = t*(Z n ) -p< s*(Z n ) < t*(Z n ) - rj. 
Consequently, using the crude bound 



J(v n+ i,g)(Z n ,R n )) + max J n +i(v n+1 , g)(Z n , s) 

s£G{Z n ) 



< 2C, 



one obtains 



J(v n+ i,g)(Z n ,r n +i^(Z n , S n )) - max J n+ i(v n+ i, g)(Z n , s) 

s6G(Z„) 



l {r n+1 ,(3(Z n ,S n )=t*(Z n )-P} 



< 2C 



L {t*(Z n )-f3<t*(Z n )- v } 



(24) 
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Now the combination of Equations (19)-(24) and Lemmas 5.5 and 5.6 yields, for all (3 < rj < 
min{A(>)} 

||^n — Vn\\ p < — Vn+l|| p + — + 2[u n +i] ||Z n+ i — Z n+ i|| p 

+\\Z n - ZnW^lv^}^ + 2C g C t * [A] x (2 + C t *C X ) 
+ (4C g C x [t*]+2[v n+1 ]^[Q])y(3[g] 1 i 



( X \\ a II [**] II ^ I 

+2Cg[ 2C\r] H — p n+ i — 5 n+ i H \\Z n — Z n \ 

" \ 7]" "P rj — p" 1 

Now suppose there exists < a < 1 such that rj = a~ 1 j3. Then the optimal choice for r\ satisfies 



2C xV 



1/ [t* 



r\ V 1 — a 



\Zn Z n \\p -\- \\S', 



n+1 



S n +1 



P / ' 



providing it also satisfies the condition < rj < min{A(z)}, Hence the result. 



□ 



Theroem 6.3 gives a recursive error estimation. Here is the initializing step. 

Theorem 6.4 Suppose the discretization parameters are chosen such that there exists < a < 1 
satisfying 



a. 



1 - a 



1/2 

\Z N _x - -^jv-illp + \\Sn - S N \\ P ) < min {A(z)}. 



Then one has 

\\Vn-i — Vn-i\ 



< || Vzv— l — Vat-i|| p + 3[g] \\Zn — Zn\\ p + o>n-i\\Zn-i — Z 



N-l\ 



+4C 9 (2C A ) i / 2 ( I L^||Z iV _ 1 - Z N - X \\ P + \\S N - S N l 



1/2 



with a N -! = [2 [g] X E 2 + 2C g C t * [A] 1 (2 + C t *C x ) + {4C g C x [t*] + 2 [ 5 ] + [Q] ) V (3 [g] X )J . 
Proof: As before, the strong Markov property of the process {X(t)} yields 



iN>r N ,g( z N-i,S N -l)} I Z N-i\ 



VN-l(Z N -x) = E[ 5 (X rjV/3(Zjv _ li5jv _ l) )l {SjN 

+ ~ E [9( Z N)l{S N <r Nl0 (Z N _ 1 ,S N - 1 )}\ Z N-l] 

= 1 {r N MZN-i,s N - 1 )>t*(z N ^ 1 )}Kg(Z N ^ 1 ) 

+ 1 {r N , l3 (z N ^ 1 ,s N - 1 )<f(z N . 1 )}J(9,g){Z N -i,r N) p(Z N ^i,S N -i)). 

The rest of the proof is similar to that of the previous theorem. 



□ 



As in Section |5j it is now clear that an adequate choice of discretization parameters yields 
arbitrarily small errors if one uses the stopping-time tjv- 
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7 Example 

Now we apply the procedures decribed in Sections [4] and [6] on a simple PDMP and present 
numerical results. 

Set E = [0, 1[, and dE = {1}. The flow is defined on [0, 1] by <f)(x,t) = x + vt for some 
positive v, the jump rate is defined on [0,1] by A(x) = (3x a , with (3 > and a > 1, and for 
all x G [0, 1], one sets •) to be the uniform law on [0, 1/2]. Thus, the process moves with 
constant speed v towards 1 , but the closer it gets to the boundary 1 , the higher the probability to 
jump backwards on [0, 1/2]. Figure [T] shows two trajectories of this process for xq = 0, v = a = 1 
and [3 = 3 and up to the 10-th jump. 




Figure 1: Two trajectories of the PDMP. 



The reward function g is defined on [0, 1] by g(x) = x. Our assumptions are clearly satisfied, 



and we are even in the special case when the flow is Lipschitz-continuous (see Remark A.8). All 



the constants involved in Theorems |5 . 1 1 and 6.3 can be computed explicitely. 



The real value function V$ = vq{xq) is unknown, but, as our stopping rule tn is a stopping 
time dominated by Tjv, one clearly has 

F = E X0 [g(X(r N ))] <V = sup E Xo [g(X(r))] < E xo [ sup g(X(t))] . (25) 

reM N 0<t<T N 

The first and last terms can be evaluated by Monte Carlo simulations, which provides another 
indicator of the sharpness of our numerical procedure. For 10 6 Monte Carlo simulations, one 
obtains K Xo [sup 0<t<TN g[X(t))~\ = 0.9878. Simulation results (for d = 2, xo = 0, v = a = 1, 
P = 3, up to the 10-th jump and for 10 5 Monte Carlo simulations) are given in Table [j] Note 
that, as expected, the theoretical errors decrease as the quantization error decreases. From 



equation (25), it follows 

V -V <E X0 [ sup g{X(t))] -V . 

0<t<T N 

This provides an empirical upper bound for the error. 
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Pt 


QE 


A 


V 


V 


Bi 


B 2 


#3 


10 


0.0943 


0.151 


0.7760 


0.8173 


0.1705 


74.64 


897.0 


50 


0.0418 


0.100 


0.8298 


0.8785 


0.1093 


43.36 


511.5 


100 


0.0289 


0.083 


0.8242 


0.8850 


0.1028 


34.15 


400.3 


500 


0.0133 


0.056 


0.8432 


0.8899 


0.0989 


21.03 


243.1 


900 


0.0102 


0.049 


0.8514 


0.8968 


0.0910 


17.98 


206.9 



Pt 

QE 
A 
Bi 
B 2 
B 3 



Number of points in each quantization grid 
Quantization error: QE = maxo<k<N — ©fclb 
For all z, A(z) = A 

Empirical bound E Xo [ sup < t < Tjv g(X(t))J - V 
Theoretical bound given by Theorem 
Theoretical bound given by Theorems 



5.1 



6.3 



and 



6.4 



Table 1: Simulation results. 



A Auxiliary results 

A.l Lipschitz properties of J and K 

In this section, we derive useful Lipschitz-type properties of operators J and K. The first result 
is straightforward. 

Lemma A.l Let h G L c . Then for all (x,y) E E 2 and (t,u) G one has 

h(<t>(x, t A t*(x)))e- A( - x ' tM *^ - h(<i>(y, u A t*(y))e- A ^ uM *^) 

< D l (h)\x-y\ + D 2 (h)\t-u\, 

where 

• ift< t*(x) and u < t*(y), 

D 1 (h)=[h] 1 + C h C t *[\] v D 2 (h)=[h] 2 + C h C x , 

• ift = t*(x) and u = t*(y), 

Dx(h) = [h]. + C h C t * [A] 1 + C h C x [f] , D 2 (h) = 0, 

• otherwise, 

Dtih) =[h] x + C h C t * [A] x + [h] 2 [t*] + C h C x [f] , D 2 (h) = [h) 2 + C h C x . 
Lemma A. 2 Let w & B(i^). Then for all x G E, (t,u) G one has 

J(w,g)(x,t)-J(w,g)(x,u)) < (C W C X + [g] 2 + C g C x )\t - u\, 
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Proof: By definition of J, we obtain 
J(w,g)(x,t) - J(w,g)(x,u)) 



< 



uAt*(x) 



tAt*(x) 



\Qw{<p(x,s))e- k{ - x ^di 



+ 



g(cj>(x, t A t* (x)))e~ A ^ x ' tM *^ - g((j>(x, u A t* {x)))e~ A ^ uAt * W) 



Applying Lemma A.l to h = g, the result follows. 



Lemma A. 3 Let w G L c . Then for all (x,y) G E 2 , t G R + , 

J(w,g)(x,t)-J(w,g)(y,t)) < (C W E X + [w] X E 2 + E 3 ) \x - y\ , 



where 



Ei = C x [f] +C t .[A] 1 (l + Ci.C A ), 
E 2 = C r C x [Q], 

^3 = [9] 1 + [gUt*]+c g {c t *[x] 1 + c x [t*]} 



Proof: again by definition, we obtain 
J(w,g)(x,t) - J(w,g)(y,t)) 

tAt*(x) 



< 



\Qw((j)(x,s))e- A ( x > s) di 



tAt*(y) 



\Qw{4>{y,s))e-^ y ' s) ds 



g{<f>(x, t A t*(x)))e- A ^ tAtt - g{(j){y, t A t* {y))) e - A ^ tM *^ 



Without loss of generality it can be assumed that t*(x) < t*(y). From Lemma A.l for h 
and using the fact that \t At*(x) — t At*(y)\ < \t*(x) - t*(y)\, we get 



J(w,g)(x,t) - J(w,g)(y,t)) 

tAt*(x) 



< 



XQw(<f>(x, s))e- A ^ s) - XQw{4>{y, s))e~ AM 
+ (C w C x [t*} +E 3 )\x-y\. 
By using a similar results as Lemma |A.l| for h = XQw, we obtain the result. 
Lemma A. 4 Let w G L c . Then for all (x, y) G E 2 , 

\Kw(x) - Kw(y)\ < {C W E 4 + [w] x E 2 + [w)^[Q))\x - y\, 
where E 4 = 2C X [t*] + C t * [X] 1 (2 + C t * C x ) . 

Proof: The proof is similar to the previous ones and therefore omitted. 



ds 
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A. 2 Lipschitz properties of the value functions 

Now we turn to the Lipschitz continuity of the sequence of value functions (v n ). Namely, we 
prove that under our assumptions, v n belongs to L c for all < n < N. We also compute the 
Lipschitz constant of v n on E as it is much sharper in this case than [v n ] 1 , see Remark 



2.2 



We start with proving sharper results on operator J. 
Lemma A. 5 Let w E L c . Then for all x E E and (s, t) E 



sup J(w,g)(x,u) - sup J(w,g)(x,u) < (C W C\ + [g] 2 + C g C\)\t - s|, 

u>t u>s 

Proof: Without loss of generality it can be assumed that t < s. Therefore, one has 

sup J(w, g)(x, u) — sup J(w, g)(x, u) = sup J(w, g)(x, u) — sup J(w, g)(x, u). (26) 

u>t u>s u>t u>s 

Remark that there exists t £ [t A t* (x) , t* (x)] such that sup J(w, g)(x, u) = J(w,g)(x,t). Conse- 

u>t 

quently, if i > s then one has sup J(w, g)(x, u) — sup J(w, g)(x, u) 

u>t u>s 

Now if I € [t A t*(x), s[, then one has 

sup J(w, g)(x, u) — sup J(w, g)(x, u) < J(w, g)(x, t) — J(w, g)(x, s). 



From Lemma |A.2[ we obtain the following inequality 



sup J(w,g)(x,u) - sup J(w,g)(x,u) < (C W C\ + [g] 2 + C g C\)\t - s\ 

u>t u>s 



Combining equations (26), (27) and the fact that \t — s < \t — s\ the result follows. 



(27) 



□ 



similarly, we obtain the following result. 
Lemma A. 6 Let w G L c . Then for all (x,y) E E 2 , 

sup J(w,g)(x,t)- sup J(w,g)(y,t) < (C W E 5 + [w] X E 2 + E 6 ) \x - y\, 

t<t*(x) t<t*(y) 

where E 5 = E 1 + C x [t*] and E 6 = E 3 + ([g] 2 + C g C x ) [t*] . 

Now we turn to (v n ). Recall from [11^ that for all < n < N, (v n ) is bounded with C Vn = C g . 



Proposition A. 7 For all < n < N, v n E L c and 

[v n ] 1 < e CxCt * (2 [vn+i] X E 2 + C g E x + C g E 4 + C g C t * [A] 1 (l + C x C t * )) 

+e c ^{([g] 1+ [g] 2 [t*))v([v n+1 )M)}, 
[v n } 2 < e c ^*{c g C x {A + C x C t *) + [g] 2 ], 

[Vn] < [Vn+l]^ + C g E b + {E & V 



(28) 
(29) 
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Proof: Clearly, vn = g is in L c . Assume that t> n +i is in L c , then by using the semi-group 
property of the drift it can be shown that for any x £ E, t £ [0,t*(x)], one has (see Eq. 
(8)]) 



v n ((f>(x,t)) = e A( - x ' t U(supJ(v n+ i,g)(x,u)\/Kv n+ i(x))-Iv n+ i(x,t)\. 

L u>t J 

Remark that for x £ E, t £ M + , one has 

sup J(v n+1 ,g)(x,u) y Kv n+1 {x) < sup J(v n+ i, g)(x, u) V Kv n+1 (x) = v n (x). 

u>t u 

Set (x, y) G -E 2 and t G [0, i*(ar) A i*(y)]. It is easy to show that 

e A(x, t )_ e A(,, t )| < e^^c^^ 

- Iv n+ i(y,t) < (c Vn+1 Ei + [ Vn+l\ ^E2 \\x y\. 



(30) 
(31) 

(32) 
(33) 



Then, equations (30)-(33) yield 



v n (</>(x,t))-v n (<f>(y,t)) < [\v n (x)\ + \lv n+1 (x,t)\}e CxCt * [^-^CtAx ~ V\ 

+ e A( - y ^\ sup J(v n+ i,g)(x,u) - J(v n+ i, g)(y, u) V \Kv n+ i(x) - Kv n+ i(y) \ \ 

+ e A ^(a n+lJ E 1 +[t; n+1 ] 1 ^ 



|x-y|. 

For x £ E, t £ [0,t*(x)] and n G N, note that 

e A(x,t) < e c A c t% (/^(^t)! < C A C„ n+1 C t *, and \v n+1 (x)\ < C g . 



(34) 



(35) 



Therefore, we obtain inequality (28) by using equations (34), (35) and Lemma A. 3 A. 5 and 
the fact that C g E 1 + E 3 = C g E A +[g] 1 +[g] 2 [t*] • 

Now, set x £ E and t, s £ [0,t*(x)]. Similarly, one has 



e A(x,t) _ e A(x,s) 



< e CxCt *Cx\t- s\ 



Iv n+ i(x,t) - Iv n+ i(x,s) < C\C Vn+1 \t-s\ 



(36) 
(37) 



Combining equations (30), (plft, (36) and (37), it yields 



t - s 



■j n ((j)(x,t))-vMx,s)) \ < [\v n (x)\ + \lv n+1 (x,t)\}e c * Ct *C x \i 

+ e A(x,t) \ gup j( Vn+ug )(x iV \ - sup J(v n+1 ,g)(x,u) +C x C Vn+1 \t- s\\. (38) 

^ U>t 1t*>R ' 



u>s 



Finally, inequality (29) follows from equations (35), (p8| and Lemma A. 4 



One clearly has [vn\ < [v n ] 1 + [ u n] 2 [**]' Finally, set (x,y) £ E~ . By definition, one has 
\v n {x) - v n (y)\ 



< 



sup J(v n+ i,g)(x,u) - sup J(v n+ i,g)(x,u) V \Kv n+ x(x) - Kv n+ i(y)\ 

u<t*(x) u<t*(y) 



and we conclude using Lemmas 



A.6 



and 



A.4 



and the fact that £4 = E5 + Cf [A] 1 . 



Remark A. 8 Note that \y n \ is much sharper than [v n ] 1 - If in addition to our assumpt 
the drift (ft is Lips chitz- continuous in both variables, then with obvious notation, one has 
[vn] [(/>]■ for i £ {1,2,*}, which should yields better constant, see the example in Section 



□ 

ions, 
Vn] { < 
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B Structure of the stopping times of PDMP's 

Let r be an {^i} tEK -stopping time. Let us recall the important result from M.H.A. Davis [6] 



Theorem B.l There exists a sequence of nonnegative random variables (-R„) ngN „ such that R n 
is ^t„_i -measurable and r A T n+ \ = (T n + R n +i) A T n+ \ on {r > T n }. 

Lemma B.2 Define R\ = R\, and R k = R0-< Sk 1< R k 1 i- Then one has 

oo 

t = y^ y R n A S n . 

n=l 

Proof: Clearly, on {T k < r < T k+ i}, one has Rj > Sj and R k +i < S k +i for all j < k. 
Consequently, by definition Rj = Rj for all j < k + 1, whence 

oo k oo 

Y,Rnf\S n = ^Rn f\S n + {R k+1 AS k+1 } + Rnf\S n 
n=l n=l n=k+2 

oo 

= Tfc + Rk+l + ^ -Rn A S n . 

n=k+2 

Since -Rfc+i = Rk+i < Sfc+i we have Rj = for all j > k + 2. Therefore, XmLi A S n = 
Tfc + .Rfc+i = r, showing the result. □ 

There exists a sequence of measurable mappings ( r fc) fcgN defined on E x (R + x E) ^ with 
value in M + satisfying 

i?i = ri(Z ), 

Rk = r *(^o,rn)) 

where T k = (S\, Z x , . . .,S k ,Z k ). 

Definition B.3 Consider p £ N*. Let (-Rfc) fcgN a sequence of mappings defined on E x 
(R + x x f2 with value in M + defined by 

Rl{y,l,u) = r p+ i(y,7), 

and for k > 2 

flfc(y,7,w) = r p+fc (y,7,r fc _i(u;))l {5fc _ i < i?fe _ i} (y,7,a;). 
Proposition B.4 Assume that T p < r < T/v- Then, one has 

t = Tp + ?(Z ,r p ,8 Tp ), 
where t: E x (R + x x f2 — > M + is defined by 

t(v,J,u) = ^4(!/,7,w)AS„(u). (39) 

n=l 
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Proof: First, let us prove by induction that for fc£N„ one has 

Rk{Zo,T p ,QT p ) = Rp+k- 



(40) 



Indeed, one has Ri(Zq, T p , 9t p ) = R P +i, and on the set {r > T p }, one also has R p +i = R p +i- 
Consequently, R\{Zq,T p ) = Rp+i- Now assume that R k (Zo,T p ,9T p ) = R P + k - Then, one has 

R k+ i{Z {uj),T p (uj),9 Tp (uj)) = r p+fc+ i(Zo(w),rp(a;),r fc (6'T p (^)))l {5fe <^ fe} (^o(w),rp(w),6' Tp (w)). 

By definition, one has T k (9T p (u>)) = (S p+ i(u>), Z p+ i(co), . . . , S p+k (uj), Z pJrk (uj)} and the induction 
hypothesis easily yields l^ Sk ^ k y(Z (u;),T p (uj),9 Tp (uj)) = 1 

{Sp+k^Rp+k}^^' Therefore, we get 

R k+1 (Z ,T p ,9 Tp ) = Rp+k+i, showing (40). 



Combining equations ( 39 ) and ( 40 ) yields 

t(Zq,T p , 9t p ) = y~] Rp+n A S p+n . 



N-n 



(41) 



71=1 



However, we have already seen that on the set {T > T p }, one has R k = R k > S k , for k < p. 



Consequently, using equation (41), we obtain 

v 



N N 

T p + r(Z ,T p ,9 Tp ) = J2Sk+ Rk/\S k = Y,Rk/\S k . 

k=l k= P +l k=l 



Since r < Tjy, we obtain from Lemma 
result. 



B.2 



N 



and its proof that r = ^^Rn A S n , showing the 

□ 



n=l 



Proposition B.5 Let (k«) n6N * be a sequence of nonnegative random variables such that U n 
J r T„- 1 -meO'Surable and U n+ \ = on {S n > U n }, for all n 6 N*. Set 



is 



U = J2u n AS n . 



n=l 



Then U is an {j^} K -stopping time 



Proof: Assumption 2.1 yields 



{U < t} = U 



n=0 



{T n < u < T n+1 } n{u < t} n{t < T n+1 ] 
u({T n < u < T n+1 }n{u < t}n{T n+1 < t} 



(42) 



From the definition of U n , one has {U > T n } = {U n > S n }, hence one has 

{T n < u < T n+1 } n{u < t} n{t < T n+1 } ={s n < u n } n{T n + u n+1 < t] 

n{T n <t}n{t<T n+1 } 
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Theorem 2.10 ii) in [8. now yields {S n < U n } C\{T n + U n +i < t} n{T n < t} £ Tt, thus one has 

{T n < U < T n+1 } n{U < t} n{t < T n+1 } e T t . (43) 
On the other hand, one has 

{T n < u < T n+1 } n{u < t} n{r n+ i <t} = {s n < u n } n{u n+1 < s n+ i} n{T n+ i < t] 

Hence Theorem 2.10 ii) in [8] again yields 

{T n < U < T n+ i} n{U < t} n{T n+1 < t} G Tf (44) 



Combining equations (42), ( |43| ), and ( |44[ ) we obtain the result. □ 

Corollary B.6 For any (y, 7) G E x (K + x E) p , r(y,j, .) is an {^t} tgK+ - stopping time satis- 
fying r(y,7, .) < T/v-p. 

Proof: It follows form the definition of that Rk(y, 7, ^) < Sk(w) implies 7, a;) = 

and the nonnegative random variable i?fc(y,7, .) is J r T fc _i -measurable. Therefore, Proposition 



B.5| yields that r(y,j, .) is an {^ r t} t6K+ -stopping time, finally, by definition of r, see equation 

iV-p 

, one has r(y, 7, .) < S'n = T/v- p showing the result. □ 
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